RAID-CUBE: The Modern Datacenter Case for RAID

“Big Data” processing in modern datacenters dramatically increases the data volume moving between applications and storage. A major challenge is achieving acceptable levels of availability and reliability in an environment characterized by huge storage capacities, large numbers of disk drives, and very high interconnection bandwidth (e.g., 100 petabytes and 17000 disk drives at CERN1). In this paper, we show that existing RAID mechanisms are insufficient, and that the mean time to data loss (MTTDL) falls drastically as the number of disks and data volume increase. We introduce a new high availability storage configuration, which we call RAID-CUBE, and show that it is more resilient to data loss as the datacenter scales in capacity than existing RAID dual parity and triple parity schemes. We also identify the limits to capacity of a datacenter (in terms of the number of disks) to maintain an acceptable MTTDL for different data protection mechanisms. Finally, we briefly introduce an effective mechanism for bit error protection for large sequential IOs in this environment.